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\Q [ Abstract 

After Shannon, entropy becomes a fundamental quantity to de- 
^^ ' scribe not only uncertainity or chaos of a system but also information 

"r-3 ■ carried by the system. Shannon's important discovery is to give a 

Qh! mathematical expression of the mutual entropy (information), infor- 

■y | mation transmitted from an input system to an output system, by 

j^ ■ which communication processes could be analyzed on the stage of 

mathematical science. In this paper, first we review the quantum mu- 
;-~j tual entropy and discuss its uses in quantum information theory, and 

secondly we show how the classical mutual entropy can be used to 
S^ , analyze genomes, in particular, those of HIV. 

1 Introduction 

The study of mutual entropy (information) and capacity in classical system 
was extensively done after Shannon by several authors like Kolmogorov [IE] 
and Gelfand [U]] • In quantum systems, there have been several definitions of 
the mutual entropy for classical input and quantum output [""J HU 112 UZj ■ in 
1983, the author denned [""3| the fully quantum mechanical mutual entropy 
by means of the relative entropy of Umegaki [3*1] . and it has been used 
to compute the capacity of quantum channel for quantum communication 
process; quantum input-quantum output ["""""."""""*. 



A correlated state in quantum systems, so-called quantum entangled state 
or quantum entanglement, are used to study quantum information, in partic- 
ular, quantum computation, quantum teleportation, quantum cryptography 
00E1I3IHIIS1I2S1IIII1III2]- Recently Belavkin and Ohya g] characterized 
the entangled states and introduced the mutual entropy for entangled states 
to measure the degree of the entanglement. 

In part I of this paper, we mainly discuss the following two topics; (1) the 
quantum mutual entropy, the capacity of quantum channel and their uses 
in quantum communication; (2)the quantum mutual entropy for entangled 
states. 

Genome sequences is considered to carry information, and the information 
is stored in base or amino acid sequences so that it originates the life itself. In 
part II of this paper, we present how information theory is used to investigate 
the "information" stored in DNA. In particular, we shall discuss the uses of 
several informations (entropies) and the artificial codes to analyze genomes 
of, for instance, HIV. 



Part I 

Quantum Information 

2 Quantum Mutual Entropy 



The quantum mutual entropy was introduced in jzoj for a quantum input and 
quantum output, namely, for purely quantum channel, and it was generalized 
for a general quantum system described by C*-algebraic terminology[2n]- We 
here review the quantum mutual entropy in usual quantum system described 
by a Hilbert space. 

Let H be a Hilbert space for an input space, B(7i) be the set of all 
bounded linear operators on TC and S(H) be the set of all density operators 

on Ti. An output space is described by another Hilbert space 7i, but often 

7i =7i. A channel from the input system to the output system is a mapping 

A* from S{TL) to S(TL) [22j. A channel A* is said to be completely positive 
if the dual map A satisfies the following condition: ££,- =1 AlA(B%Bj)Aj > 

for any n6N and any A,- E B(H), Bj E B(H). 



An input state p e S(TC) is sent to the output system through a channel 
A*, so that the output state is written as p = A* p. Then it is important to 
ask how much information of p is correctly sent to the output state A* p. This 
amount of information transmitted from input to output is expressed by the 
mutual entropy in Shannon's theory. 

In order to define the quantum mutual entropy, we first mention the 
entropy of a quantum state introduced by von Neumann pSD . For a state p, 
there exists a unique spectral decomposition 

p = SfcAfcPfc, (1) 

where Xk is an eigenvalue of p and Pk is the associated projection for each Afc. 
The projection Pk is not one-dimensional when Xk is degenerated, so that the 
spectral decomposition can be further decomposed into one-dimensional pro- 
jections. Such a decomposition is called a Schatten decomposition, namely, 

P = SfcAfcEfc, (2) 

where E k is the one-dimensional projection associated with A^ and the de- 
generated eigenvalue A/, repeats dimP fc times. This Schatten decomposition 
is not unique unless every eigenvalue is non-degenerated. Then the entropy 
(von Neumann entropy) S (p) of a state p is defined by 

S(p) = -trplogp, (3) 

which equals to the Shannon entropy of the probability distribution {A^} : 

S(p) = -£> fe logA fc . ( 4 ) 

k 

The quantum mutual entropy was introduced on the basis of the above 
von Neumann entropy for purely quantum communication processes. The 
mutual entropy depends on an input state p and a channel A*, so it is denoted 
by I (p; A*), which should satisfy the following conditions: 

(1) The quantum mutual entropy is well-matched to the von Neumann 
entropy. Furthermore, if a channel is trivial, i.e., A* = identity map, then 
the mutual entropy equals to the von Neumann entropy: I (p; id) = S (p). 

(2) When the system is classical, the quantum mutual entropy reduces to 
classical one. 

(3) Shannon's fundamental inequality 0< I (p; A*) < S (p) is held. 



Before mentioning the quantum mutual entropy, we briefly review the 
classical mutual entropy. Let (fi, J 7 ) , (f2,jF)be an input and output mea- 
surable spaces, respectively, and P (fi) , P (Jij are the corresponding set of 
all probability measures (states). A channel A* is a mapping from P (Q) to 
P (flj and its dual A is a map from the set B (fl) of all Baire measurable 
functions on Q to B (Q) . For an input state /i e P (fl) , the output state pZ = 
A*/i and the joint state (probability measure) $ is given by 

$(Q* Q)= [_A(l Q )dfi, QeF,QeT, (5) 

Jq 

where 1q is the characteristic function on fi : \q (u) = < „ / a ry\ • The 
classical entropy, relative entropy and mutual entropy are defined as follows: 

S Ox) = sup J - £> (40 !og/i (^) ; (M e V (O) I , (6) 

5(/i,z/) = sup|x;/i(^)log^|;{MeP(fi)j, (7) 

J( yU ;A*)=5($,^®A», (8) 

where V (Q) is the set of all finite partitions on Q, that is, {Ak} G V (Q) iff 
4 6f with A k n Aj- = (k ^ j>nd U^ =1 A fc = Q. 

In order to define the quantum mutual entropy, we need the joint state 
(it is called "compound state" in the sequel) describing the correlation be- 
tween an input state p and the output state A*p and the quantum relative 
entropy. A finite partition of Q in classical case corresponds to an orthogonal 
decomposition {E k } of the identity operator I of H in quantum case because 
the set of all orthogonal projections is considered to make an event system 
for a quantum system. It is known [2n]that the following equality holds 

sup < - ^ trpE k log trpE k ; {E k } \ = -trp log p, 

and the supremum is attained when {E k } is composed of the Schatten de- 
composition of p. Therefore the Schatten decomposition is used to define the 
compound state and the quantum mutual entropy. 



The compound state Oe (corresponding to joint state in CS) of p and A*p 
was introduced in fI3\ 121] , which is given by 

o E = J2 XkEk ® A * E ^ ( 9 ) 

k 

where E stands for a Schatten decomposition of p, so that the compound 
state depends on how we decompose the state p into basic states (elementary 
events), in other words, how to see the input state. 

The relative entropy for two states p and a is defined by Umegaki 
and Lindblad CHI, which is written as 



„, ^_ J trp (log p — log a) (when ranp C r ana) , , 

1 oo (otherwise) 

Then we can define the mutual entropy by means of the compound state 
and the relative entropy [23], that is, 

I (p; A*) = sup {S {B E , P ® A*p) ; E = {E k }} , (11) 

where the supremum is taken over all Schatten decompositions. Some com- 
putations reduce it to the following form: 

I (p; A*) = sup | J2 X ^S (A*E k , A*p) ; E = {E k } I , (12) 

This mutual entropy satisfies all conditions (1)~(3) mentioned above. 

When the input system is classical, an input state p is given by a prob- 
ability distribution or a probability measure, in either case, the Schatten 
decomposition of p is unique, namely, for the case of probability distribution 

; p = {A fc }, 

P = ^A fc 5 fe , (13) 

k 

where 5 k is the delta measure, that is, 

h(j)=h, 3 = {l\ k h = 4^J. (14) 

Therefore for any channel A*, the mutual entropy becomes 

I{p;A*) = Y,*kS{A*6 k ,A*p), (15) 

k 
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which equals to the following usual expression of Shannon when it is well- 
defined: 

I (p; A*) = S (A*p) - Y, ^S (A*4) , (16) 

k 
which has been taken as the definition of the mutual entropy for a classical- 
quantum(-classical) channel |U EJ [TTJ [T2J HI] . 

Note that the above definition of the mutual entropy (2.12) is written as 



/(p;A*) 

= sup < ^2 ^kS (A*p k , A*p) ;p = y^ X k pk e F (p) \ , 
I is fc J 

where F D (p) is the set of all orthogonal finite decompositions of p |5U] . 

More general formulation of the mutual entropy for general quantum sys- 
tems was done [23 El i n C*dynamical system by using Araki's or Uhlmann's 
relative entropy|3J |33J I2E]- This general mutual entropy contains all other 
cases including measure theoretic definition of Gelfand and Yaglom |1(J| . 



3 Communication Processes 

The information communication process is mathematically set as follows: 
M messages are sent to a receiver and the kth message uj( k > occurs with 
the probability A&. Then the occurrence probability of each message in the 
sequence (uj^ 1 \u!^- 2 \ ■ ■ •,to'^ M - ) )of M messages is denoted by p = {Xk} ? which 
is a state in a classical system. If £ is a classical coding, then £ (u) is a 
classical object such as an electric pulse. If £ is a quantum coding, then £ (u) 
is a quantum object (state) such as a coherent state. Here we consider such a 
quantum coding, that is, £ (u>( k ') is a quantum state, and we denote £ (u>( k ') 
by Ok- Thus the coded state for the sequence (u/ 1 - ) ,u/ 2 - ) , ■ ■ -,0/*^) is written 
as 

o- = ^A fc a fe . (17) 

k 

This state is transmitted through a channel 7, which is expressed by a com- 
pletely positive mapping V* from the state space of X to that of X , hence 



the output coded quantum state a is Y*a. Since the information transmission 
process can be understood as a process of state (probability) change, when 

fl and Q are classical and X and X are quantum, the process (3.1) is written 

as 

p(n) ^s(n) -^ s(n) ^ p(n), (is) 

where S* (resp.S ) is the channel corresponding to the coding £ (resp.£ ) and 
S (Ti) (resp.iS(7i) ) is the set of all density operators (states) on 7i (resp.7i 

)• 

We have to be care to study the objects in the above transmission process 

(3.1) or (3.3). Namely, we have to make clear which object is going to study. 

For instance, if we want to know the information capacity of a quantum 

channel 7(= T*), then we have to take X so as to describe a quantum system 

like a Hilbert space and we need to start the study from a quantum state in 

quantum space X not from a classical state associated to a message. If we like 

to know the capacity of the whole process including a coding and a decoding, 

which means the capacity of a channel £070 £(= E o T* o S*), then we have 
to start from a classical state. In any case, when we concern the capacity of 
channel, we have only to take the supremum of the mutual entropy / (p; A*) 
over a quantum or classical state p in a proper set determined by what we 
like to study with a channel A*. We explain this more precisely in the next 
section. 

4 Channel Capacity 

We discuss two types of channel capacity in communication processes, namely, 
the capacity of a quantum channel T* and that of a classical (classical- 
quantum-classical) channel S o PoH*. 

(1) Capacity of quantum channel: The capacity of a quantum channel 
is the ability of information transmission of a quantum channel itself, so 
that it does not depend on how to code a message being treated as classical 
object and we have to start from an arbitrary quantum state and find the 
supremum of the quantum mutual entropy. One often makes a mistake in this 
point. For example, one starts from the coding of a message and compute 
the supremum of the mutual entropy and he says that the supremum is the 

7 



capacity of a quantum channel, which is not correct. Even when his coding 
is a quantum coding and he sends the coded message to a receiver through 
a quantum channel, if he starts from a classical state, then his capacity is 
not the capacity of the quantum channel itself. In his case, usual Shannon's 
theory is applied because he can easily compute the conditional distribution 
by a usual (classical) way. His supremum is the capacity of a classical- 
quantum-classical channel, and it is in the second category discussed below. 
The capacity of a quantum channel T* is defined as follows: Let Sq(c 
S(TC)) be the set of all states prepared for expression of information. Then 
the capacity of the channel T* with respect to So is defined by 

C s °(T*) = sup{I(p;T*);peS }. (19) 

Here I (p; T*) is the mutual entropy given in (2.11) or (2.12) with A* = T*. 
When S = S(H) , C S W (r*) is denoted by C (T*) for simplicity. The 
capacity C (T*) is written as 

C(r*) = svL-p{I(p;r*);peS(H)}, (20) 

where the supremum is taken over all states p with its orthogonal pure de- 
composition J2k ^kPk of p. In J27[[2T], we also considered the pseudo-quantum 
capacity C p (T*) defined by (4.1) with the pseudo-mutual entropy I p (p;T*) 
where the supremum is taken over all finite decompositions instead of all 
orthogonal pure decompositions: 

L p (p; r *) = SU P \ Yl XkS ( r * Pfc ' F * p ) ' p = 5Z Xkpk ' 

V k k 

finite decomposition} . (21) 

However the pseudo-mutual entropy is not well-matched to the conditions 
explained in Sec. 2, and it is difficult to be computed numerically. The relation 
between C (r*) and C p (T*) was discussed inj^Zj. From the monotonicity of 
the mutual entropy 26J, we have 

o < c So (r ) < cf (r ) < sup {S(p) ; P es }. 

(2) Capacity of classical- quantum- classical channel: The capacity of C-Q- 

C channel 5 o V* o S* is the capacity of the information transmission process 
starting from the coding of messages, therefore it can be considered as the 

8 



capacity including a coding (and a decoding). As is discussed in Sec. 3, an 
input state p is the probability distribution {A*,} of messages, and its Schatten 
decomposition is unique as (2.9), so the mutual entropy is written by (2.11): 



llp;E o r*oS* 

= J2 X kS (s* o r* o E*8 k , E o r* o E*p\ . (22) 

k ^ ' 

If the coding E* is a quantum coding, then E*5 k is expressed by a quantum 
state. Let denote the coded quantum state by a k and put a = E*p = 
J2k ^k^k- Then the above mutual entropy is written as 

/ L- e o r* o ~*\ = J2 x kS (e o rv fe , e o rv") . (23) 

This is the expression of the mutual entropy of the whole information trans- 
mission process starting from a coding of classical messages. Hence the ca- 
pacity of C-Q-C channel is 

C Po (e o T* o E*\ = sup{/ (p; E o r o E*\ ; p E P }, (24) 

where Po(c -P(^)) is the set of all probability distributions prepared for 
input (a-priori) states (distributions or probability measures). Moreover the 
capacity for coding free is found by taking the supremum of the mutual 
entropy (4.4) over all probability distributions and all codings E*: 

c p (Z* o r*) = sup{/ L Eoro~*Y,pe P , E*}. (25) 

The last capacity is for both coding and decoding free and it is given by 

C% ( r ) = sup{/ L EoT*o~*y,pe P , E*, E}. (26) 

These capacities Cf , C^ do not measure the ability of the quantum channel 
T* itself, but measure the ability of T* through the coding and decoding. 



Remark that J2k XkS(T*ak) is finite, then (4.4) becomes 

/ U e o r* o e* j = 5(s* o rv) - ^ A fe 5(i* o rv fc ). (27) 

Further, if p is a probability measure having a density function /(A) and each 
A corresponds to a quantum coded state cr(A), then & = J /(A) cr(A)c?A and 



J I p; H o r* o E* 
= S(E o TV) - / f(X)S(E o r*a(X))d\. (28) 

This is bounded by 

S(V*a)-Jf(X)S(V*a(X))d\, 

which is called the Holevo bound and is computed in several occasions plot 127] 
The above three capacities C p °, Cf , C cd ° satisfy the following inequalities 



< C Po ( E o T* o E* I < C c Po ( E o F 

<c3(r)<Bup{5(p);peP } 

where 5*(p) is not the von Neumann entropy but the Shannon entropy: - 
X^AfelogAfc. 

The capacities (4.1), (4. 6), (4. 7) and (4.8) are generally different. Some 
misunderstandings occur due to forgetting which channel is considered. That 
is, we have to make clear what kind of the ability (capacity) is considered, the 
capacity of a quantum channel itself or that of a classical-quantum(-classical 
) channel. The computation of the capacity of a quantum channel was carried 
in several models in [2JJ EH| 

5 Quantum Entanglements 

Recently the quantum entangled state has been mathematically studied [HI 
El HH] > hi which the entangled state is defined by a state not written as a form 
Ylk ^kPk ® Ok with any states pk and Ok- A state written as above is called a 
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separable state, so that an entangled state is a state not belonged to the set of 
all separable states. However it is obvious that there exist several correlated 
states written as separable forms. Such correlated states have been discussed 
in several contexts in quantum probability such as quantum filtering |3], 
quantum compound state [23] , quantum Markov state [T] and quantum lifting 
J2J. In jB], we showed a mathematical construction of quantum entangled 
states and gave a finer classification of quantum sates. 

For the (separable) Hilbert space AT of a quantum system, let A = B (/C) 
be the set of all linear bounded operators on /C. A normal state p on A 
can be expressed as p (A) = trgK^An, A & A, where Q is another separable 
Hilbert space, k is a linear Hilbert-Schmidt operator from Q to K, and k^ is 
the adjoint operator of k from K, to Q. The (unique) density operator a G A 
associated to the state tp : p (A) = trAa, A G A, is written by k such as 
a = kk) . This k is called the amplitude operator, and it is called just the 
amplitude if Q is one dimensional space C, corresponding to the pure state 
p(A) = k^Ak, for a k G /C with tvn = \\n\\ 2 = 1. In general, Q is not one 
dimensional, the dimensionality dim£ must be not less than dima/C. 

Since Q is separable, Q is realized as a subspace of Z 2 (N) of complex 
sequences (i.e.,(* = (£ n ) , ( n G C, n G N with Yl IC"I < +°°)j so that 
any vector (' = (( n ) represents a vector ( = J2( n \n) in the standard basis 
{|n>} G £ of/ 2 (N) . 

Given the amplitude operator k, one can define not only the states a = 
kk^ and p = k^k on the algebras A{= B (/C)) and B{= B ((/)) but also an 
entanglement state on the algebra £> ® A. of all bounded operators on the 
tensor product Hilbert space Q ® K by 

9 (S ® A) = trgB^An = tr K AnBn ] 

for any I? G B. This state is pure as it is the case of T = C in the theorem 
below, and it satisfies the marginal conditions: For any B G B,A G A, 

Q(B(g)I)= tr g Bp, (I ® A) = frjcAr. 

Theorem 1 ||/Lei 9:S®i^Cfca state 

(5 ® A) = tr^ f (B ® A) V, (29) 

defined by an amplitude operator ip on a separable Hilbert space £ into the 
tensor product Hilbert space Q ®K, ; ip : £ — > Q ®K with trj^yib = 1. TTien 

11 



there exists an amplitude operator k : Q — > T ® /C snc/i £/ia£ £/je state G can 
fre achieved by an entanglement 

9 (5 <g> A) = trgBrf (I <g> A) « = tr>^ (7 <g> A) «£«* (30) 

TVie entangling operator k is uniquely defined up to a unitary transformation 
of the minimal space T . 

The entangled state (5.2) is written as 

(B ® A) = trgBcf) (A) = tr K A<j>* (B) , (31) 

where (A) = k^ (I <S> A) k is in the predual space £>* C B of all trace-class 
operators in £/, and 0* (5) = tr^KBiv is in .4.* C *4. The map is the 
Steinspring form of the general completely positive map A — > £>*, written in 
the eigen-basis (|n)} of Q C I 2 (N) of the density operator p = (J) as 

0(A) = J^|m)/4,(J<g> A) Kn (n|, Ae A (32) 

where n n is the vector in !F®K such that k = J2 n K n(n\. The dual operation 
0* is the Kraus form of the general completely positive map B — > A*, given 
in this basis as 

0, (5) = J] (n| £ \m) tr^K n Kl, B e B. (33) 

It corresponds to the general form of the density operator 

O4, = ^2 \ n )( m \ ® tr T K n n} m (34) 

m,n 

for the entangled state G with the weak orthogonality property 

Definition 1 The dual map 0* : B — ► A* to a completely positive map 
: A — > B*, normalized as trg(f)(I) = 1, is called the quantum entanglement 
of the state p — (I) on B to the state a — 0* (I) on A. The entanglement 
by (A) = o~ l l 2 Ao- l l 2 of the state p = a on the algebra B = A given by the 
standard entangling operator k = a x l 2 is called standard. 

12 



A compound state, playing the similar role as the joint input-output 
probability measures in classical systems, was introduced in (23] as explained 
in Sec. 2. It corresponds to a particular diagonal type 

n 

of the entangling map (|3*3*)l in the eigen-basis(Schatten decomposition) of the 
density operator p = Y^Pn\n)(n\. Therefore the entangled states, generaliz- 
ing the compound state, also play the role of the joint probability measures. 

The diagonal entanglements can be considered as a quantum correspon- 
dences of symbols {1, • • • , n, • • • } to quantum states. The general entangled 
states are described by the density operators 9$ of the form (J34J) which 
is not necessarily diagonal in the eigen-representation of the density opera- 
tor p = J2 n Pn\ n )( n \- Such nondiagonal entangled states were called in [23] 
the quasicompound (q-compound) states, so we can call also the nondiago- 
nal entanglement the quantum quasi- correspondence (q-correspondence) in 
contrast to the d- correspondences, described by the diagonal entanglements, 
giving rise to the d-compound states. 

Take trjrK n ^ n = v n v\, v n G /C. The density operator 

9 = ^2\ n ) ( n \ ® °"n, °n = PnV n V ] n (36) 

n 

define the compound states on B ® A, giving the quantum correspondences 
n \— > \n)(n\ with the probabilities p n . The entanglement with (J36|) is a 
diagonal entanglement such as 

0* (B) = ^2p n (n\B\n)v n vl (37) 

whose dual is 

(f){A) = J2Pn\n)vlAv n {n\. (38) 

n 

These entanglements has the stronger orthogonality 

trjrK n K ] m = p n v n vl5™, (39) 

for the amplitudes n n e T <g> /C of the decomposition k = J2 n K n (n\ in com- 
parison with the weak orthogonality of n n in (|34|) . 

13 



Definition 2 The positive diagonal map 

</>,(B) = ^2(n\B\n)a n (40) 

into the subspace of trace-class operation K with trg<j)* (I) = 1, is called quan- 
tum d- entanglement with the input probabilities p n = tr^a n and the output 
states u) n = Pn X o~ n , and the corresponding compound state (GJ) is called d- 
compound state. The d- entanglement is called c- entanglement and compound 
state is called c-compound if all density operators a n commute: a m a n = o n a m 
for all m and n. 

Note that due to the commutativity of the operators B <g> I with I <g> A 
on Q ® /C, one can treat the correspondences as the nondemolition measure- 
ments in B with respect to A. So, the compound state is the state prepared 
for such measurements on the input Q. It coincides with the mixture of 
the states, corresponding to those after the measurement without reading 
the sent message. The set of all d-entanglements corresponding to a given 
Schatten decomposition of the input state p on A is obviously convex with 
the extreme points given by the pure elementary output states u n on A, cor- 
responding to a not necessarily orthogonal decompositions a = J2 n a n m -^° 
one-dimensional density operators a n = p n u n . 

The orthogonal Schatten decompositions a = J2 n Pn^n correspond to the 
extreme points of c-entanglements which also form a convex set with mixed 
commuting u n for a given Schatten decomposition of a. The orthogonal 
c-entanglements were used in to construct a particular type of Accardi's 
transition expectations jT] and to define the entropy in a quantum dynamical 
system via such transition expectations jH]- 

Thus we classified the entangled states into three categories, namely, q- 
entangled state, d-entangled state and c-entangled state, and their rigorous 
expressions were given. 

6 Mutual Entropy via Entanglements 

Let us consider the entangled mutual entropy by means of the above three 
types compound states. We denote the quantum mutual entropy of the 
compound state achieved by an entanglement 0* : B — >• A* with the 
marginals 

Q(B(g)I)= tr g Bp, (I ® A) = tr K Aa (41) 
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by Iff, (p, a) or 1$ (A, B) and it is given as 

I4 (p, a) = tr6^ (log Of - log {p®cr)). (42) 

Besides this quantity describes an information gain in a quantum system 
(A, a) via an entanglement </>* with another system (B,p), it is naturally 
treated as a measure of the strength of an entanglement, having zero the value 
only for completely disentangled states (}4*Tj) . corresponding to 9</> = p <8> o . 

Definition 3 The maximal quantum mutual entropy for a fixed state a 

H a (A) = sup{/ (A, B) ■ 0* (I) = a} (43) 

is called q- entropy of the state a. The differences 

H (f) (B\A) = H a (A)-I (t) (A,B), 
D 4 ,{B\A) = S{a)-I 4 ,{A,B) 

are respectively called the q-conditional entropy on B with respect to A and 
the degree of disentanglement for the compound state </>. 

H,p {B\A) is obviously positive, however D^ {B\A) has the positive maxi- 
mal value S (a) = sup {D^ (B\A) ; 0* (/) = a} and can achieve also a negative 
value 

inf {Z> (B\A) ; ^ (I) = a} = S (a) - H a (A) (44) 

for the entangled states [S], which is called the chaos degree in [IB*]. 

Let us consider Q as a Hilbert space describing a quantum input system 
and /C as its output Hilbert space. A quantum channel A* sending each 
input state defined on Q to an output state defined on K. A deterministic 
quantum channel is given by a linear isometry T :Q — >/C with T^T = J 
(J is the identify operator in Q) such that each input state vector r\ G Q, 
\\t]\\ = 1 is transmitted into an output state vector T// e /C, ||Tr/|| = 1. The 
mixtures p = J2 n VW^n of the pure input states ui n = r/nr/l are sent into the 
mixtures o = 'Yl in p n o'n with pure states a n = TuJnT^ . A noisy quantum 
channel sends pure input states us into mixed ones o = A*uj given by the 
dual of the following completely positive map A 

A (A) = T f (h ® A) T, AeA (45) 
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where T is a linear isometry from Q to J-\®1C, T^ (Ii ® /) T = I , and I\ is the 
identity operator in a separable Hilbert space T\ representing the quantum 
noise. Each input mixed state p e B (Q) is transmitted into the output state 
o = A*p on^CB (X), which is given by the density operator 

a = tr^T P T^ = A*pe A*. (46) 

We apply the proceeding discussion of the entanglement to the above 
situation containing a channel A*. For a given Schatten decomposition p = 
Yl n Pn\ n )( n \ an d the state a = A*p,we can construct three entangled states 
of the proceeding section: 

(1) q-entanglement (pi and q-compound state Q\ are given as 



<f>l(B) = y^ (n | B | to) tr^nK^ 

n,m 

e l = ^2 \ n )( m \ ® tr f K « K L 

with the marginals p = X] n Pnl n )( n l? °" — ^*p = ^ r G^ an d 't r K. K n l A n = 
p n w n <5™ = K^Kn for u; n = A*|n)(n|. Let £ g be the convex set of all completely 
positive maps <f) q . 

(2) d- entanglement <j) d and d-compound state Q d , are given as 



1(B) =J2(n\B\n) tr T K n K\ 

n 



with the same marginal conditions as (1). Let 8^ be the convex set of all 
completely positive maps (f) d . 

(3) c-entanglement 4>1 and c-compound state 9^ are same as those of (2) 
with commuting {u n } . Let S c be the convex set of all completely positive 
maps <p c . 

Now, let us consider the entangled mutual entropy and the capacity of 
quantum channel by means of the above three types of compound states. 
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Definition 4 The mutual entropy I q (p, A*) and the q-capacity C q (A*) for a 
quantum channel A* are defined by 

I q (p, A*) = sup {S(8l,p® A* p);^e£ g }, (47) 

C g (A*)=sup{I,(p,A*);p}. 

The d-mutual entropy, the d-capacity and the c-mutual entropy, the c-capacity 
are defined as above using 8i and 9 C ,, respectively. 

Note that due to £ c C £ d C £ q , we have the inequalities 

I q (p,A*)>I d (p,A*)>I c (p,A*), 
C q (A*)>C d (A*)>C c (A*) 

for a deterministic channel (A* = id), the two lower mutual entropies coincide 
with the von Neumann entropy: 

I d (p, id) = -trp logp = I c (p, id) . 

The capacity for such a channel is finite if A has a finite rank, C d (A*) < 
dim/C. On the other hand, the q-mutual entropy can achieve the q-entropy 

I q (p,id) = -2trp\ogp 

and its capacity is bounded by the dimension of the algebra A, C q (A*) < 
dim A which doubles the d-capacity dim/C when A = B (/C) . These equalities 
will be related to the work on entropy by Voiculescu |35J. 



Part II 

Information Genetics 

7 Entropy Evolution Rate 

Genome sequence carries information as an order of four bases, and the 
information is transmitted to m-RNA, which makes a protein as a sequence 
of amino acids by a help of t-RNA. 
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In information theory, the concept of information has two aspects, one 
of which expresses the amount of complexity of a whole system like a se- 
quence itself and another does the structure of the system(or message) such 
as the rule stored in the order of sequence |13j. From Shannon's philosophy, a 
system has the larger complexity, the system carries the larger information, 
from which the information of a whole system has been expressed by the 
entropy. The structure of the system is studied in the field named " coding 
theory", that is, how to code the messages is essential in communication of 
information. 

Pioneering works for application of information theory to genome se- 
quence were done by Smith [48 j and Gatlin[H7j, since then few works have been 
appeared along this line. In 1989 143.,, I introduced a measure representing 
the difference of two genome or amino acid sequences, which is called the 
entropy evolution rate and has been used to make phylogenetic trees |4^1l4T] . 
The coding theory was applied to the study of genome sequences in order to 
examine the coding structure of several species [4*4"|. 

Let A and B be amino acid or base sequences. When they are considered 
to be close each other, for instance, they specify an identical protein, we first 
have to align these sequences by inserting a gap "*", whose arrangement 
is called the alignment of sequences [4T} E21 HE]- As an example, take two 
sequences A and B given as 

A : a c b a c d 
B : adbcacb 

Then the aligned sequences become 

A : acb * acd 
B : adbcacb 

After the alignment, two sequences have the same length. Take two 
aligned sequences A and B having the length n given by A=(ai : a2, • ••, 
a n ), 5=(bi : b2, • • •, b n ), where a^ bj are the gap * or an amino acid for 
an amino acid sequence or a base for a base sequence. There are 21 events 
(20 amino acids and *) in an amino acid sequence and 5 events (4 bases 
and *) in a base sequence. Therefore, in an aligned sequence, the occurrence 
probability of each amino acid (resp. base) is associated, and it is denoted 
by pk for k-th amino acid (resp. base), where < k < 20 (resp. < k < 4) 
and "0" corresponds to the gap. Then the entropy (information) carried by 
the amino acid (resp. base) sequence A is defined as 



S(A)(oi S(p)) = -J^P^ogPk 
k 

where p denotes the probability distribution (p k ) . Similarly, there exists the 
event system (B, q = (q k ) ) for the amino acid (or base) sequence B, and 
its entropy is denoted S(B) or S(q). Through the alignment, we can find 
the correspondence between the amino acid (resp. base) of A and that of 
B, which enables to make the compound event system (A x B, r) of A and 
B. Here r is the joint probability distribution between A and B, so that it 
satisfies £) fc r jk = pj and £) fc r jk = q k . 

The most important information measure in Shannon's communication 
theory is the mutual entropy (information) expressing the amount of infor- 
mation transmitted from (A,p) to (B,q), which is defined as follows: 



/(A £) = $>,* log 



Tik 



» 



Using the entropy and the mutual entropy, an quantity measuring the simi- 
larity between A and B was introduced as 

i r/(A,B) , /(A,*) 



2 [ S(A) S(B) 

which was called the symmetrized entropy ratio or the entropy evolution 
rate in 43jand it takes the value when A and B are completely different 
and 1 when they are identical. The minus of this rate from 1 indicates the 
difference between A and B. We here call it the entropy evolution rate, and 
it is denoted by p(A, B) : 

p(A,B) = l-r(A,B). 

Using this rate, we can construct a genetic matrix and write a phylogenetic 
tree of species (331 E]- Note that a similar measure providing the difference 
between A and B can be defined as 



S(A) + S(B)-I(A,B) , 

but this dose not have a precise meaning from the information theoretical 
point of view. 
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An application of this rate to the variation of HIV virus for six patients 
reported by [SU EE1 HB H3is discussed in 02|. 

8 Code Structure of Genes 

When we send an information (a series of messages), we have to process the 
messages in proper forms so as to correctly and quickly send the information 
to a receiver. It is the coding theory that teaches us how to process the 
messages properly. There are many ways to encode the messages in commu- 
nication processes. We shall explain some of such codings and their use to 
the study of genome sequences. 

Let i = («i,«2, ■ ■ -,ik) be a properly processed information sequence. In 
order to send the symbol j to a receiver correctly, that is, to avoid some 
noise and loss in the course of information transmission, we have to add 
some redundancy (parity check symbol) p = (pi,P2, 4 • ',Pn-k) to the infor- 
mation symbol i. This redundancy p detects or corrects the errors in the 
communication process. The whole code-word now becomes 

X = (»l,»2, • • -,ik,Pl,P2,- ■ -,Pn-k)- 

The above x is called a systematic code, and to make the systematic code x 
from the information symbol i is called a coding. A coding is realized by a 
Galois group GF(q) with a primary number q and a certain parity check p. 
When the relation between i and p is linear, the code so obtained is called a 
linear code. Among the linear codes, there are the block code such as cyclic 
code and BCH code and the convolutional code such as self-orthogonal code 
and Iwadare code. Each code has its own parity check correcting the error 
such as random error, burst error and bite error. We do not go into the details 
of the coding theory here, but we explain how to use the coding technique 
to examine the code structure of genome sequences. 

When we like to know the code structure of a species, an organism, a spe- 
cial part of a genome sequence indicating a protein or a set of these objects, 
we rewrite a base sequence of an object into the sequence of the symbols of 
GF{2 2 ) because we have four bases, and we apply several coding methods to 
the symbol sequence and get the coded symbol sequence (systematic code), 
then we write it back the coded base sequence. This process is written as 
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follows: 

Base sequence A => Symbol sequence A s 
=> Coded symbol sequence Ajj =>■ Coded base sequence A c 

In order to know the common code structure of the sequences A\A^ ■ ••, A 
we use the following index obtained from the entropy evolution rate and a 
coding C applied to the sequences: 



Tl ■ 



D 



c 



{En l ZU + M A ^ A i)-p( A ?> A ?)\} 



nCj 



where n C2 is the combination 2 out of n, that is, n C2 = 2 ~ and A4 is an 
amino acid sequence or a base sequence. Note that when A{ is originally an 
amino acid sequence, we first translate it the corresponding base sequence 
and take the above procedure, then we convert the coded base sequence to 
the coded amino acid sequence. If this index D c is close to 0, then a common 
code structure of the group {iii 2 , • ■■,A n } is close to the structure of the 
code C used. 

We studied the code structure of Vertebrate, Onco virus and HIV virus 
by means of the structure index Dc- We used some parts of the base sequence 
for each organisms; MDH, LDH, hemoglobin a, (3 for Vertebrate; pol, env, 
gag for Onco and HIV virus. Then we obtained the following results: 

(1) Vertebrate has a similar code structure of the convolutional code with 
high ability correcting the burst errors like the codes named UI, ZI, and the 
code structure of hemoglobin a is closest to that of the artificial codes. 

(2) Onco virus has a similar code structure of the cyclic code with the 
burst error correction (C2) or the self-orthogonal code (TB,VD), so that it 
does not have so high ability correcting the errors. 

(3) HIV virus has a similar code structure of the cyclic code (CI) or the 
self-orthogonal code with the random error correction (TA) , so that the 
ability correcting the errors is low. 

(4) In Onco and HIV virus, the pol protein has the closest code structure 
of the artificial codes. 

The structure index is applied to the study of the variation and the con- 
dition of the patients having the HIV infection in |5Uj . 
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